Metacognitive Adaptation to Enhance Lifelong Language Learning

Lifelong language learning (LLL) aims at new tasks and retaining old in the field of NLP. LAMOL is a recent LLL framework following data-free constraints. Previous works have been researched based on with additional computing more time costs or parameters. However, they still gap between multi-task (MTL), which regarded as upper bound LLL. In this paper, we propose Metacognitive Adaptation (Metac-Adapt) almost without adding cost computational resources to make model generate better pseudo samples then replay them. Experimental results demonstrate that Metac-Adapt par MTL better.